Combining a high resolution narrow field display and a mid resolution wide field display

ABSTRACT

A head mounted display (HMD) includes a first display portion included in the HMD, the first display portion having a first pixel density, a second display portion included in the HMD, the second display portion having the first pixel density, a third display portion attached to the HMD, the third display portion having a second pixel density, and at least one image combiner configured to combine two images by reflecting an image projected by the first display portion and the second display portion and allowing an image projected by the third display portion to pass through the at least one image combiner.

RELATED APPLICATION

This application claims priority to and the benefit of U.S. ProvisionalPatent Application No. 62/115,455, filed on Feb. 12, 2015, entitled“COMBINING A HIGH RESOLUTION NARROW FIELD DISPLAY AND A MID RESOLUTIONWIDE FIELD DISPLAY”, the contents of which are incorporated in theirentirety herein by reference.

FIELD

Embodiments relate to virtual reality (VR) head mounted displays (HMD).

BACKGROUND

For a HMD system to provide an optimal foveal resolution over ahorizontal field of a human eye, a display may include upwards of 9,000columns and/or rows (e.g., a so-called 9K display) or more across theentire display.

SUMMARY

According to one general aspect, a head mounted display (HMD) includes afirst display portion included in the HMD, the first display portionhaving a first pixel density, a second display portion included in theHMD, the second display portion having the first pixel density, a thirddisplay portion attached to the HMD, the third display portion having asecond pixel density, and at least one image combiner configured tocombine two images by reflecting an image projected by the first displayportion and the second display portion and allowing an image projectedby the third display portion to pass through the at least one imagecombiner.

According to another general aspect, a head mounted display (HMD) of avirtual reality (VR) system includes a first combined display systemconfigured to receive a first image and a second combined display systemconfigured to receive a second image the second image being a differentview perspective of the first image. The first combined display systemincludes a first display portion configured to project a first portionof the first image, a second display portion configured to project asecond portion of the first image, and an image combiner configured tocombine two images by reflecting the first portion of the first imageand allowing the second portion of the first image to pass through theimage combiner. The second combined display system includes a thirddisplay portion configured to project a first portion of the secondimage, a fourth display portion configured to project a second portionof the second image, and an image combiner configured to combine twoimages by reflecting the first portion of the second image and allowingthe second portion of the second image to pass through the imagecombiner.

According to still another general aspect, a head mounted display (HMD)includes a first display portion configured to project a first image, asecond display portion configured to project a second image, and animage combiner configured to combine two images by reflecting the firstimage and allowing the second image to pass through the image combiner.

Implementations can include one or more of the following features. Forexample, the at least one image combiner can be further configured toblock a portion of the image projected by the third display portion, theportion of the image projected by the third display portioncorresponding to the image projected by the first display portion andthe second display portion. The at least one image combiner can befurther configured to block a portion of the image projected by thefirst display portion and the second display portion, the portion of theimage projected by the first display portion and the second displayportion corresponding to the image projected by the third displayportion.

For example, the first pixel density can be a higher pixel density thanthe second pixel density, and the third display portion can beconfigured to reduce a brightness of a portion of the image projected bythe third display portion, the portion of the image projected withreduced brightness by the third display portion corresponding to theimage projected by the first display portion and the second displayportion. The first pixel density can be a lower pixel density than thesecond pixel density, and the first display portion and the seconddisplay portion can be configured to reduce a brightness of a portion ofthe image projected by the first display portion and the second displayportion, the portion of the image projected with reduced brightness bythe first display portion and the second display portion correspondingto the image projected by the third display portion.

For example, the two images can be blended at a boundary between the twoimages. The HMD is communicatively coupled to a computing deviceconfigured to generate the two images using an optical fiber. The firstpixel density can be a higher pixel density than the second pixeldensity, and the first display portion and the second display portioncan be positioned above the third display portion. At least one of thefirst display portion, the second display portion and the third displayportion can include a curved portion. The HMD can include at least onelens. The HMD can include at least one of a motion sensor and an eyetracking component each configured to detect a change in a view positionof a user of the HMD.

For example, the HMD can be communicatively coupled to a computingdevice associated with an image repository via an optical fiber. Thefirst display portion and the second display portion can have a firstpixel density, the third display portion and the fourth display portioncan have a second pixel density, the second pixel density being lowerthan the first pixel density, and the third display portion and thefourth display portion can include a curved portion. The first displayportion and the second display portion can have a first pixel density,the third display portion and the fourth display portion can have asecond pixel density, the second pixel density being lower than thefirst pixel density, and the first display portion and the seconddisplay portion are positioned above the third display portion and thefourth display portion.

For example, the first display portion can be configured to reduce abrightness of a portion of the first portion of the first image, and thethird display portion can be configured to reduce a brightness of aportion of the first portion of the second image. The second displayportion can be configured to reduce a brightness of a portion of thesecond portion of the first image, and the fourth display portion can beconfigured to reduce a brightness of a portion of the second portion ofthe second image.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will become more fully understood from the detaileddescription given herein below and the accompanying drawings, whereinlike elements are represented by like reference numerals, which aregiven by way of illustration only and thus are not limiting of theexample embodiments and wherein:

FIGS. 1A, 1B and 1C illustrate block diagrams of a head mounted display(HMD) according to at least one example embodiment.

FIG. 2A illustrates a schematic representation of visual fields.

FIG. 2B illustrates the frontal plane and the midsagittal frontal plane.

FIG. 2C illustrates a plane orthogonal to the frontal planes thatbisects the eyes. Also shown are the gaze vectors from the eyes to apoint A.

FIGS. 3A, 3B and 3C illustrate a block diagram of a side view of a twodisplay HMD according to at least one example embodiment.

FIGS. 3D and 3E illustrates a block diagram of a front view of a twodisplay HMD of FIGS. 3A and 3B according to at least one exampleembodiment.

FIG. 4 illustrates a block diagram of a top view of the two display HMDof FIGS. 3A and 3B according to at least one example embodiment.

FIGS. 5A, 5B and 5C illustrate a block diagram of a side view of anothertwo display HMD according to at least one example embodiment.

FIG. 5D illustrates a block diagram of a front view of a two display HMDof FIGS. 5A and 5B according to at least one example embodiment.

FIG. 6 illustrates a block diagram of a top view of the two display HMDof FIGS. 5A and 5B according to at least one example embodiment.

FIGS. 7A and 7B illustrate a block diagram of a side view of stillanother two display HMD according to at least one example embodiment.

FIG. 7C illustrates a block diagram of a front view of a two display HMDof FIGS. 7A and 7B according to at least one example embodiment.

FIGS. 8, 9 and 10 illustrate block diagrams of a top view of alternateconfigurations of the two display HMD of FIGS. 7A and 7B according to atleast one example embodiment.

FIG. 11 illustrates method associated with a two display HMD accordingto at least one example embodiment.

FIG. 12 illustrates a block diagram of a virtual reality VR systemassociated with a HMD according to at least one example embodiment.

FIG. 13 illustrates a technique that uses a light guide to relay imagesthrough a thin glass or plastic plate to create an augmented realitydisplay system according to at least one example embodiment.

FIG. 14 illustrates a cross sectional view of another embodiment of anoptical system in which two displays are used per eye to create an imageof high resolution in the central field, and reduced resolution in thefields peripheral the central field according to at least one exampleembodiment.

FIG. 15 illustrates an internal reflection to make a portion of thelight guide transparent according to at least one example embodiment.

FIG. 16 illustrates a plan view of a Fresnel lens and a light guideaccording to at least one example embodiment.

FIG. 17 illustrates a top view of a block diagram of a portion of a HMDincluding a hybrid lens system according to at least one exampleembodiment.

FIGS. 18 and 19 illustrate views of a line diagram of a HMD according toat least one example embodiment.

It should be noted that these Figures are intended to illustrate thegeneral characteristics of methods, structure and/or materials utilizedin certain example embodiments and to supplement the written descriptionprovided below. These drawings are not, however, to scale and may notprecisely reflect the precise structural or performance characteristicsof any given embodiment, and should not be interpreted as defining orlimiting the range of values or properties encompassed by exampleembodiments. For example, the relative thicknesses and positioning ofregions and/or structural elements may be reduced or exaggerated forclarity. The use of similar or identical reference numbers in thevarious drawings is intended to indicate the presence of a similar oridentical element or feature.

DETAILED DESCRIPTION OF THE EMBODIMENTS

While example embodiments may include various modifications andalternative forms, embodiments thereof are shown by way of example inthe drawings and will herein be described in detail. It should beunderstood, however, that there is no intent to limit exampleembodiments to the particular forms disclosed, but on the contrary,example embodiments are to cover all modifications, equivalents, andalternatives falling within the scope of the claims.

In order to instill a feeling of presence and to instill a sense ofbelief while using a HMD, video imagery that matches human perceptioncan be desirable. Human perception may require approaching a horizontalfield of view (FOV) of about 150 degrees per eye for high peripheralvision. For the HMD system to provide foveal resolution over an entirehorizontal field of 150 degrees, the required display may include 9,000columns (e.g., a so-called 9K display). For an HMD system including a 9Kdisplay, if it were available, may necessitate the use of a large datarate, would have high power consumption, would be expensive, and mayalso contribute too an excessive weight for the HMD. Accordingly, an HMDsystem configured to display video imagery that matches human perceptionwithout use of a 9K display can be desirable.

FIG. 1A illustrates a block diagram of a head mounted display (HMD)according to at least one example embodiment. As shown in FIG. 1A, theHMD 100 includes a first display 105 and a second display 110. The firstdisplay 105 can have a first pixel density and the second display 110can have a second pixel density. For example, the first display 105 canbe a mid resolution display having pixel density in the range of 250 to800 pixels per inch (ppi) and the second display 110 can be a highresolution display having pixel density higher than 800 ppi. In otherwords, the first display 105 can have a pixel density less than a pixeldensity of the second display 110. The second display 110 can beconfigured to display images and/or video data in a field of visionassociated with a high resolution region of binocular overlap (describedin more detail below). The first display 105 may be configured todisplay images and/or video in a field of vision outside of, orperipheral to, the high resolution region of binocular overlap,including a further region of binocular overlap including a lowerresolution. Two displays in the HMD of FIG. 1A may be configured todisplay three-dimensional (3D) and/or stereographic images.

FIG. 1B illustrates a block diagram of a head mounted display (HMD)according to at least one example embodiment. As shown in FIG. 1B, theHMD 150 includes a first display 105-L, a second display 105-R, a thirddisplay 110-L and a fourth display 110-R. The first display 105-L andthe second display 105-R can be formed as a single display. As such, thefirst display 105-L and the second display 105-R can each be a portionof a single display. Further, the third display 110-L and the fourthdisplay 110-R can be formed as a single display. As such, the thirddisplay 110-L and the fourth display 110-R can each be a portion of asingle display. Accordingly, the first display 105-L, the second display105-R, the third display 110-L and the fourth display 110-R can bereferred to as the first display portion 105-L, the second displayportion 105-R, the third display portion 110-L and the fourth displayportion 110-R.

The first display 105-L and the second display 105-R can have a firstpixel density and the third display 110-L and the fourth display 110-Rcan have a second pixel density. For example, the first display 105-Land the second display 105-R can be mid resolution displays having pixeldensity in the range of 250 to 800 pixels per inch (ppi) and the thirddisplay 110-L and the fourth display 110-R can be high resolutiondisplays having pixel density higher than 800 ppi. In other words, thefirst display 105-L and the second display 105-R can have a pixeldensity less than a pixel density of the third display 110-L and thefourth display 110-R. The third display 110-L and the fourth display110-R can be configured to display images and/or video data in a fieldof vision associated with a high resolution region of binocular overlap(described in more detail below).

The first display 105-L and the second display 105-R may be configuredto display images and/or video in a field of vision outside of, orperipheral to, the high resolution region of binocular overlap,including a further region of binocular overlap including a lowerresolution. The first display 105-L and the third display 110-L can beassociated with a left eye and display a left eye image in a 3D image orvideo. The second display 105-R and the fourth display 110-R can beassociated with a right eye and display a right eye image in the 3Dimage or video. In an alternative embodiment, the first display 105-Land the second display 105-R are formed from one integral display panelcapable of showing an image that is partitioned into two partscomprising left and right images. The HMD of FIG. 1B may be configuredto display three-dimensional (3D) and/or stereographic images.

FIG. 1C illustrates a block diagram of a head mounted display (HMD)according to at least one example embodiment. As shown in FIG. 1C, theHMD 175 includes a first display 105 and a second display 110. The firstdisplay 105 can have a first pixel density and the second display 110can have a second pixel density. For example, the first display 105 canbe a mid resolution display having pixel density in the range of 250 to800 pixels per inch (ppi) and the second display 110 can be a highresolution display having pixel density higher than 800 ppi. In otherwords, the first display 105 can have a pixel density less than a pixeldensity of the second display 110. The second display 110 can include atleast one portion 110-A such that when images projected by the at leastone portion 110-A are combined with images projected by the firstdisplay 105 a display including display portion 110-L and displayportion 110-R is formed. The HMD of FIG. 1C may be configured to displaythree-dimensional (3D) and/or stereographic images.

FIG. 2 illustrates a schematic representation from a top view ofhorizontal visual fields. As shown in FIG. 2, eyes 205-1, 205-2 (e.g.,human eyes), including pupils 230-1, 230-2, can visually perceive a leftvisual field 210 and a right visual field 215. Within the left visualfield 210 and the right visual field 215, the eyes 205-1, 205-2 canvisually perceive a full binocular overlap visual field 220 which may beas large as 120 deg. A sub-region of full binocular overlap is shown asa high resolution region of binocular overlapping visual field 225. Asshown in FIG. 2B (and throughout this description), a vertical planewhich we term the midsagittal frontal plane bisects the head between theeyes, and a vertical plane that we call the vertical frontal planeintersects the head orthogonal to the midsagittal plane at a positionthat bisects the eyes 205-1 and 205-2. FIG. 2C shows a horizontal planethat extends in a direction left and right (or horizontally) withrespect to the eyes 205-1, 205-2 and that also bisects the eyes. We callthe plane in FIG. 2C the horizontal frontal plane. The three planesdefined in FIGS. 2B and 2C intersect at the midpoint of a line segmentextending from the center of the left eye to the center of the righteye.

The fovea is the central portion of the retina of each of the eyes205-1, 205-2 that perceives the highest resolution. The direction ofgaze (illustrated by vector G parallel to the midsagittal plane) may bedefined by a vector from the center of the fovea through the center ofthe pupil. Neither eye 205-1 nor eye 205-2 turns or rotates withsufficient comfort to allow the direction of gaze to scan the fullhorizontal visual field 210 or 215. Therefore, imagery beyond thecomfortable turning limit of the eyes 205-1, 205-2 will not be viewed bythe fovea (although such imagery will be viewed by other parts of theretina). Accordingly, peripheral images provided by a VR system do notneed to be at foveal resolution. As a result, a high resolution (e.g., a9K) display projected onto the retina across the entirety of the lefthorizontal visual field 210 and the right horizontal visual field 215may not be necessary or desirable (i.e., technically or costprohibitive). Accordingly, example embodiments can utilize two displayswithin an HMD for a VR system. One of the two displays (e.g., the seconddisplay 110) can be configured to display images and/or video in thehigh resolution region of binocular overlap visual field 225 that willbe projected on or near the fovea. Another of the two displays (e.g.,the first display 105) can be configured to display images and/or videooutside of, or peripheral to, the high resolution region of thebinocular overlap visual field 225. In other words, a display can beconfigured to display images and/or video in the regions of the leftvisual field 210 and the right visual field 215 and the full binocularoverlap visual field 220 that are not inclusive of the high resolutionregion of binocular overlap visual field 225.

It should be noted that although the fovea subtends only a small arc,the rotation of the eyes can extend the range of angles over which adisplay should match foveal resolution. When the user's eyes move andthe direction of gaze changes, such as when reading, resolution matchingthe fovea is desirable over the range of comfortable gaze scanning. Therange of comfortable gaze scanning is approximately 15 degrees in anydirection with respect to vector G in FIG. 2A. The gaze can scan overlarger angles with progressively more discomfort as the scan angleincreases beyond 15 degrees from the mid-sagittal plane.

In an example implementation of an HMD (while referring to FIG. 2C), allhorizontal angles (e.g., angles along the horizontal frontal plane suchas θ) can be measured with respect to the mid-sagittal frontal plane(i.e. the plane of head symmetry centered between the eyes. The leftvisual field 210 and the right visual field 215 regions represent thevisual fields of the left and right eyes that can be supplied imagesfrom a low or medium resolution display (or displays) with partialbinocular overlap that can be as much as 120 degrees per eye (e.g., thefull binocular overlap visual field 220 region, matching the overlap inthe human visual system). The high resolution region of binocularoverlap visual field 225 can be supplied by left and right highresolution displays with 100% overlap. Accordingly, a high resolutionbinocular image (e.g., as displayed using high resolution displays, thesecond displays 110 or the second displays 325 described below) can havea field of view (FOV) of, for example, 60 degrees which is approximatelydouble the FOV needed to account for comfortable eye motion. In thisway, foveal resolution (e.g., 1 minute of arc) can be provided at doublethe range of comfortable eye motion (from 15 to +15 degrees), and alower resolution image (e.g., as displayed using a lower resolutiondisplay, the first display 105 or the first display 320 described below)can be provided beyond 60 degrees. If two low or medium resolutionimages provide 150 degrees FOV to the left and right eyes, with anoverlap of 120 degrees, then the combined total FOV can be about 180degrees. In other words, the first display 105 (or the first display 320described below) could be provided in the form of left and rightdisplays which when combined are configured to provide a combined FOV ofabout 180 degrees.

Other implementations are within the scope of this disclosure. Forexample, the high resolution image (e.g., as displayed using the seconddisplay 325 described below) can have a FOV of, for example, a rangebetween 40 and 90 degrees. The two low or medium resolution images(e.g., as displayed using a lower resolution display, the first display105 or the first display 320 described below) can have a complementary(to the high resolution image) FOV range in order to cover a combinedtotal FOV of about 180 degrees.

FIGS. 3A and 3B illustrate a block diagram of a side view of a twodisplay per eye HMD 300 according to at least one example embodiment. Asshown in FIG. 3A, a combined display system 330 includes a lens 310, animage combiner 315, a first display 320 and a second display 325 (allshown in cross section). An eye 305 (one is shown, but it is understoodthat the HMD includes a matching system for the other eye) visuallyperceives the images (illustrated as 320-a and 325-a in FIG. 3C) of thecombined display system 330 as a single image including the image fromthe first display 320 and the image from the second display 325 as shownin FIG. 3B. Away from the eye 305 is labelled as distal, close to theeye 305 is labelled as proximate. Top and bottom are labelled as adescriptive convenience. However, top and bottom could be switched witha corresponding alteration of component positions.

In an example implementation, the first display 320 and the seconddisplay 325 can be different in size so as to subtend different fieldsof view. The resultant corresponding images 320-a and 325-a subtenddifferent fields of view, as shown in FIG. 3C. As such, two smaller highresolution displays (one for each eye) can be used, for example, tosubtend a central field of view of about 60 degrees (e.g., to create thehigh resolution region of binocular overlap visual field 225), and lowerresolution larger displays can be used, each to subtend an angle of 150degrees (e.g., in the regions including the high resolution region ofbinocular overlap visual field 225 and beyond). In other words, thefirst display 320 can be a larger, lower resolution display and thesecond display 325 can be a smaller, higher resolution display. In orderto preserve the high resolution image 325-a produced by display 325,pixels in display 320 can be set to black if their images overlay pixelsin the image 325-a of display 325. In other words, display 320 may showa black rectangle in the region of high resolution so that its imagedoes not interfere with the image of display 325. Alternatively, theimages may be arranged so that low resolution features in region 225 areprovided by the lower resolution display 320, and higher resolutionfeatures in region 225 are provided by high resolution display 325.

The term resolution in the context of the display emitting area refersto the number of pixels in a given unit of surface area, or the densityof pixels (or pixel density). The higher resolution display would have ahigher density of pixels or pixel density than the lower resolutiondisplay so that in the combined virtual image, the pixels of the higherresolution display subtend a smaller angle than the pixels of the lowerresolution display. The resolution (or density) of pixels is typicallyexpressed as pixels per inch

In example implementations, the first display 320 (e.g., low or mediumresolution display) can be used to extend the horizontal field of viewto the left and right of, as well as above and below, the second display325 (e.g., high resolution display). In some implementations, thedisplay (e.g., the first display 320), pixels near the temporal ends arefar from the center of the optical axis (vector G). As a result, imagedistortions and image aberrations can become more noticeable (and insome cases severe). These aberrations and distortions are generally afunction of distance from the optical axis, becoming more severe as thepixels are located at gaze angles farther from the optical axis.Accordingly, the lens 310 may be configured to correct for thesedistortion and aberrations. The lens 310 may be, for example, a toric oran aspherical lens. Although in cross section a simple double convexlens is shown, the lens may have any shape and/or may be a Fresnel lens.Additionally, the image may be inversely distorted by the softwaregenerating the pixel data before the data are supplied to the display,so that distortion by the lens cancels the inverse distortion, yieldingan undistorted viewable image.

The image combiner 315 may be configured to combine (e.g., opticallycombine) an image 320-a from the first display 320 and an image 325-afrom a second display 325. For an HMD built for two eyes which canrequire four displays (as shown in FIG. 1B), the resultant image can beof a high resolution in a region of the high resolution region ofbinocular overlap visual field 225 (produced by the two high resolutiondisplays) and of a lesser resolution in at least one region peripheralto the high resolution region of binocular overlap visual field 225,produced by at least one of the medium resolution displays.

In the example implementation shown in FIG. 3A, the image combiner 315may be configured to reflect light rays emitted from the second display325 toward the lens 310 (e.g., by orienting the image combiner at anangle (e.g., 30, 45, 60 and/or the like degrees) between the seconddisplay 325 and the lens 310). Further, the image combiner 315 may beconfigured to allow light rays emitted from the first display 320 topass through the image combiner 315 toward the lens 310. The imagecombiner 315 may be configured to reflect (substantially or partiallyreflect) light associated with an image as transmitted from the seconddisplay 325 toward the lens 310. Further, the image combiner 315 may beconfigured to allow light to pass through (substantially or partiallypass through), the light being associated with an image as transmittedfrom the first display 320 toward the lens 310.

In an example implementation, the image combiner 315 may be configuredto reflect a portion of a projected image (e.g., a portion of theprojected photons or a portion of the luminance representing the image)such that less than all of the image is reflected and/or the intensityof the light associated with all of the image may be reduced with thereflection and/or the intensity of the light associated with one or moreportions of the image may be reduced with the reflection. Further, theimage combiner 315 may be configured to pass through a portion of animage such that less than all of the image is passed through and/or theintensity of the light associated with all of the image may be reducedand/or the intensity of the light associated with one or more portionsof the image may be reduced. Accordingly, the image combiner 315 may beconfigured to combine light associated with two images by reflecting aportion of the light associated with an image as transmitted from thesecond display 325 and allowing a portion light associated with an imageas transmitted from the first display 320 to pass through the imagecombiner 315.

In an example implementation, the image combiner 315 can operate as asemi-transparent mirror on which images can be both reflected andtransmitted. To this end, the image combiner 315 can be constructed of atransparent material (e.g., glass) with one side coated with asemi-reflective material (e.g., a thin layer of silver, aluminum,another metal, or a series of dielectric coatings). Example thin filmoptical coatings can include aluminum, gold, silver, chrome, nickel,aluminum-MgF₂, magnesium fluoride, aluminum-SiO2, silicon dioxide, zincsulfide, and the like. Combiners and/or beam splitters are commerciallyavailable that can transmit and reflect in various ratios depending onthe thickness of the coatings. As long as the reflective coating issufficiently thin (e.g., in the nano/micro meter range), portions of thephotons from each display will be both transmitted and reflected so thatthe projected images will pass through or reflect via the image combiner315. By using a coating of the appropriate thickness (e.g., in thenano/micro meter range), the brightness of the two displays may bebalanced.

The combiner may also have films designed to reflect infrared (IR) lightfrom an LED in order to make possible eye tracking. In such anembodiment, the eye is illuminated with low level IR light from a source355 near display 325. The light reflects from the combiner 315, and isdirected through the lens 310 toward the eye 305 to illuminate the eyewith IR light. The light reflected by the eye is reflected by thecombiner 315 and collected by a camera 350 placed near the display 325.In this way the combiner 315 enables a collection of an image of the eye305 and from this image the direction of gaze can be determined.

In example implementations, the first display 320 may be configured to(e.g. controlled to) reduce in intensity (e.g., brightness orluminosity) the portion of a projected image corresponding to a positionof the second display 325. Alternatively, or additionally, the imagecombiner 315 can have a portion or area configured to absorb or notallow through (e.g., apply a light absorbing material or coating) theportion of a projected image corresponding to a position of the seconddisplay 325. Accordingly, a portion of the image projected by the firstdisplay 320 may not be observable through the image combiner 315 becausethe portion of the projected image corresponding to the position of thesecond display 325 is not bright enough or is not as bright as the imageprojected by the second display 325 For example, if the intensity of thephotons from display 320 is over 100 times lower than light fromcorresponding pixels of display 325, the relative brightness will be solow as to be ignorable. Depending on the transmission of the combinerand the relative brightness of the displays, in the area of image ofdisplay 320 where it overlays the image of display 325, the image fromdisplay 325 could be 1000 times brighter. In such a case, the image fromdisplay 320, while possibly present in a minor amount, will not beobservable by the retina because the image from display 325 isoverwhelmingly brighter. Additionally, since the images are the sameexcept for resolution, the minor presence of the image from display 320is not discernable. Such an embodiment would be obtained if at thecenter of the combiner the transmission is 0.1% and the reflection is99.9%. In that case, if the displays 325 and 320 have approximately thesame brightness, the portion of the images viewed through the center ofthe combiner would have a brightness ratio of 999:1.

In some example implementations, the image combiner 315 may beconfigured to provide a transition between an image 320 a projected bythe first display 320 and an image projected by the second display 325.For example, beginning along a boundary 335 (illustrated as a dashedline in FIG. 3D) within the coating comprising the active part of theimage combiner 315 between the first display 320 and the second display325, the image combiner 315 coating may be configured to reflectprogressively less of the image projected from the second display 325 ascompared to the center of the image combiner 315. Alternatively, alongthe boundary of the image combiner 315 between the first display 320 andthe second display 325, the image combiner 315 may be configured toallow less of the image projected from the first display 320 as comparedto the outer portions of the first display 320. For example, bygradually and progressively reducing the thickness of metal coatings(e.g. by grading the coating thickness in the region 336 as shown inFIG. 3E) beginning at boundary 335 and progressing toward the edges ofthe combiner 315, the edges of a semi-transparent mirror may be hidden.The edges of the combiner 315 comprise clear glass or plastic. In thisway, the image of the low resolution display is highly suppressed nearthe center of the high resolution display, but the edges of the opticalcoating of the combiner are not evident. The edges of the combiner canbe further hidden by beveling the clear substrate.

The combiner may alternatively be formed by a polarization beam splittercoating on glass or other transparent substrate. Such coatings transmitone linear polarization and reflect the orthogonal linear polarization.If the displays 320 and 325 have light output that is linearly polarizedappropriately (such as with properly oriented LCDs), then thepolarization beam splitter may be employed as an optical combiner withless optical loss relative to a metallic film combiner. Whereas themetal film may have an optical transmission or reflection of only about49%, the polarizing film may have a transmission or reflection ofgreater than 80% for appropriately polarized light.

The presence of a high pixel density may not dictate that the highresolution display should be operated at high resolution in all regions.For example, the two images (e.g., as projected by the high resolutiondisplay and the low resolution display) can be blended at a boundarybetween the two images. By combining data on neighboring pixels, thepixel density (or resolution) can be blended so that the resolution doesnot change abruptly at the boundaries of the image of the highresolution display. It is also possible to adjust luminance and contrastso that edges are not obvious, so that as the high resolution displayfades out, the low resolution display fades in, in order to blend thetwo images into one image. This feathering may also be used to hide theboundaries of the binocular overlap region so that the eye can move fromthe binocular overlap region to the non-overlap region withoutperception of a change in brightness or contrast. In such a case,contrast and luminance feathering would be applied to both displays atthe boundary of binocular overlap.

Therefore, the first display 320 and the second display 325 may beconfigured to and/or controlled (e.g., as controlled by a video driver)to provide a transition (in addition to and/or as an alternative to theconfiguration of the image combiner 315) between an image projected bythe first display 320 and an image projected by the second display 325.For example, the first display 320 can be configured to project aprogressively darker (e.g., progressive darkening of the pixels nearingthe boundary 335) until there is a dark (e.g., black) image at thecenter of the first display 320 corresponding to the center of the imagecombiner 315. Further, the second display 325 can be configured toproject a progressively darker image (which should progressively reducereflection from the image combiner 315) from pixels of the seconddisplay 325 at some threshold margin distance in from the boundarybetween the images of the first display 320 and the second display 325,toward the edge of the second display 325 until there is a dark (orblack) image at an outer boundary of the second display 325.

In another implementation (or in combination with other implementations)pixels along the boundary can be spatially dithered. Spatial ditheringcan include alternating driving by applying control voltages to thepixels of the first display 320 and the second display 325 in proximityof the boundary. For example, for the first display 320, odd pixelsalong the horizontal and vertical axis of the boundary can be off (orblack) whereas even pixels can be based on the image. Further, for thesecond display 325, even pixels along the horizontal and vertical axisof the boundary can be off (or black) whereas odd pixels can be based onthe image.

The example configuration of the combined display system 330, the firstdisplay 320 can be the low or medium resolution display which ispositioned at the top of the combined display system 330 and the seconddisplay 325 can be the high resolution display which is positioneddistal (e.g., at a far end of the combined display system 330 whencompared to the eye 305) of the combined display system 330.Accordingly, the image of the low or medium resolution display can bepositioned behind the high resolution display (see FIG. 3B) as visuallyperceived by the eye 305.

Example embodiments can use a high resolution display (e.g., the seconddisplay 325). A high resolution display can be a display with a pixelsubtense in a virtual image plane of less than 2 minutes of arc. For abinocular overlap high resolution region (e.g., the high resolutionregion of binocular overlap visual field 225) of approximately 60degrees, a display having 1800 columns should be used to achieve 2minutes per pixel, and 3600 columns should be used to achieve 1 minuteper pixel, at the center of the field. If the vertical FOV is alsoapproximately 60 degrees, then the displays should be either 1800×1800(2 min per pixel) or 3600×3600 (1 min per pixel) in format. For mountingon the head, pixel pitch should be in the range of 5 to 15 microns,nominally 10 microns, which can result in nominal object image diagonaldimensions of between 25 mm for the 1800 pixel format and 50 mm for the3600 pixel format. A larger display can advantageously require lessmagnification.

Example embodiments can use low or medium resolution display (e.g., thefirst display 320). The low or medium resolution display can haveapproximately one tenth of the resolution of the high resolutiondisplay. Therefore, the pixel subtense at the virtual image is about 10minutes of arc. Because the full horizontal field of view for the low ormedium resolution display is about 150 degrees, the number of columnsshould be 900, depending in part on whether the display is flat orcurved. If the vertical FOV is approximately 90 degrees, then the numberof rows should be 540. If a 10 micron pixel is used to obtain 1 minuteof arc in the center of the high resolution display, then a 100 micronpixel can be used to obtain 10 min of arc in the center of the low ormedium resolution display (because the magnification is approximatelythe same for both displays). Accordingly, the low or medium resolutiondisplay can be nominally 90 mm×54 mm. The use of additional pixels orhigher pixel density can be desirable for extending the vertical fieldof view or for improving the blending of the fused images at theboundaries of the high resolution virtual image (e.g., blending along aboundary of the image combiner 315 between the first display 320 and thesecond display 325). In other words, use of additional pixels or higherpixel density can provide another mechanism to improve the transitionbetween an image projected by the first display 320 and an imageprojected by the second display 325.

In some example implementations the virtual image of the first display320 (e.g., the low or medium resolution display) may be placed at adistance d from the lens 310 and the virtual image of the second display325 (e.g., the high resolution display) may be placed at a distance hfrom the lens 310. Distance d can be a greater distance from the lens310 (for example 200 cm) than the distance h, which may be placed at anear distance to the lens 310 (such as 1 cm). Accordingly, backgroundimagery can be placed on the first display 320 (e.g., the low or mediumresolution display), and foreground imagery on the near display, thusovercoming image convergence and accommodation disparity in a threedimensional (3D) display. In other implementations, the virtual imagedistance of the two images 320-a and 325-a are the same.

FIG. 4 illustrates a schematic diagram of a top view of the two-displayHMD of FIGS. 3A and 3B according to at least one example embodiment. Asshown in FIG. 4, the first display 320 is formed as an arc or is acurved display. FIG. 4 shows a display 320 with a constant radius ofcurvature, but any complex curve may be used, including a flat surfacenear the mid-sagittal plane combined with progressively more curvaturetoward the temple. In some implementations, the display (e.g., the firstdisplay 320), toward the temporal ends, can become progressively moreoff center. As a result, image distortions and image aberrations canbecome more noticeable (and in some cases severe). Using a curveddisplay as the first display 320 (e.g., the low or medium resolutiondisplay) can reduce or minimize some of the image distortions and imageaberrations. In particular, field curvature that is introduced by theoptics can be reduced by curvature of the display.

Therefore, using a curved display as the first display 320 in additionto the lens 310 (or as an alternative to using the lens 310) can improvethe image displayed by the first display 320 particularly when viewedfrom angles of gaze not aligned with the optical axis (vector G shown inFIG. 2). In addition, using the curved display as the first display 320in addition to the lens 310 (or as an alternative to using the lens 310)can increase the field of view (FOV) of a user of the HMD. Stillfurther, the first display 320 can be a formed as straight or flat paneldisplay. In this case (as discussed above), the lens 310 may beconfigured to reduce or minimize the image distortions and imageaberrations. In another example implementation, the driver associatedwith the first display 320 can be configured to alter the image in orderto correct for the distortions.

FIGS. 5A and 5B illustrate a schematic diagram of a side cross sectionview of another two-display HMD 500 according to at least one exampleembodiment. As shown in FIGS. 5A and 5B, a combined display system 505includes the lens 310, the image combiner 315, the first display 320,the second display 325 and a faceplate 510. An eye 305 (one is shown,but it is understood that the HMD includes a matching system for theother eye) visually perceives the combined image from the display system505 as a single image including the image 320-a projected from the firstdisplay 320 and the image 325-a projected from the second display 325 asshown in FIG. 5C. The operation of the combined display system 505 issubstantially similar to the combined display system 330 with theexception of the faceplate 510.

FIG. 5D illustrates a block diagram of a front view of a two display HMDof FIGS. 5A and 5B. As shown in FIG. 5D, the second display 325 issurrounded by the border 335 which in-turn indicates a boundary betweenthe first display 320 and the second display 325. The image 320-aprojected from the first display 320 and the image 325-a projected fromthe second display 325 would project out of the page.

The faceplate 510 may be configured to convert an image projected fromthe first display 320 (as a flat surface or flat panel display) to acurved surface. For example, the faceplate 510 may be an opticalfaceplate. An optical faceplate can be (or be similar to) a coherentfiber bundle which can relay an image from a flat surface to a curvedsurface. In an example implementation, the first display 320 can be anemissive display. The use of the faceplate 510 can be desirable for usewith an emissive display because the input to the faceplate 510 can bein near contact with the imaging surface of the first display 320. Bynear contact, we mean that the distance from the pixel plane to thefaceplate is less than the pixel pitch.

FIG. 6 illustrates a block diagram of a top view of the two-display HMDof FIGS. 5A and 5B according to at least one example embodiment. Likeelements are represented by like reference numerals, which are describedabove and may not be described further with regard to this figure. Asshown in FIG. 6, the first display 320 is a flat surface or flat paneldisplay and the faceplate 510 is formed as an arc or curved surface. Theaforementioned image distortions and image aberrations can cause eyefatigue. Therefore, using a curved faceplate 510 in contact with thefirst display 320 (e.g., the low or medium resolution display) canreduce or minimize the image distortions. Therefore, using the faceplatein addition to the lens 310 can improve the image displayed by the firstdisplay 320. As a result, using a curved faceplate 510 in contact withthe first display 320 can reduce eye fatigue which can improve a userexperience.

Further, as shown in FIG. 7A, in an example implementation, the imagecombiner 315 can extend beyond the position of the second display 325 asanother mechanism to minimize visibility of its edges (illustrated aslength E). In other words, extended (in length) image combiner 315(illustrated as length E) can provide a transition between an imageprojected by the first display 320 and an image projected by the seconddisplay 325. The faceplate 510 can also be a tapered faceplate as isknown in the art. A tapered fiber optic faceplate used as an expanderhas a larger area at the exit face than at the input face. Thus thetapered faceplate may be configured to expand an image to a larger area.Accordingly, the tapered faceplate may enable use of a smaller firstdisplay 320 (e.g., a smaller, and potentially less costly, display). Theuse of tapered faceplate can be used to effect a change in the size ofan emissive display.

FIGS. 7A and 7B illustrate a schematic diagram of a side view crosssection of still another two display HMD 700 according to at least oneexample embodiment. Like elements are represented by like referencenumerals, which are described above and may not be described furtherwith regard to this figure. As shown in FIGS. 7A and 7B, a combineddisplay system 705 includes the lens 310, the image combiner 315, thefirst display 320, the second display 325 and the faceplate 510. An eye305 (one is shown, but it is understood that the HMD includes a matchingsystem for the other eye) visually perceives the combined display system705 as a single display including the first display 320 and the seconddisplay 325 as shown in FIG. 7B. The operation of the combined displaysystem 705 is substantially similar to the combined display system 330with the exception that the positions of the first display 320 and thesecond display 325 have been exchanged or transposed.

Accordingly, in the example implementation shown in FIG. 7A, the imagecombiner 315 can be configured to reflect an image projected from thefirst display 320 toward the lens 310. Further, the image combiner 315may be configured to allow an image projected from the second display325 to pass through the image combiner 315 toward the lens 310. In theexample implementation shown in FIG. 7A the second display 325 (as thehigh resolution display) is viewed directly through the image combiner315 and not by reflection from the image combiner 315. The lowerresolution display 320 is viewed by reflection from image combiner 315.As a result, imperfections in the reflective qualities of image combiner315 affect the lower resolution display 320 and interfere withresolution to a lesser degree than in embodiments in which the highresolution display is viewed by reflection. Further, the first display320 (e.g., the low or medium resolution display) can be more flexible inits configuration. For example, the first display 320 can be a flatdisplay, a curved display and/or a combination of a flat and curveddisplay.

In the example implementation shown in FIG. 7A, the image combiner maybe configured to absorb or not reflect a portion of the image projectedby the first display 320. The portion of the image that is not reflectedcorresponds to the image projected by the second display 325.Alternatively, or in addition to, the first display 320 may beconfigured to (e.g. controlled to) reduce an intensity (i.e., brightnessor luminosity) of a portion of a projected image corresponding to aposition of the second display 325. Accordingly, a portion of the imageprojected by the first display 320 may not reflect off the imagecombiner 315 because the portion of the projected image corresponding tothe position of the second display 325 is not bright enough or is not asbright as the image projected by the second display 325.

FIGS. 8, 9 and 10 illustrate schematic diagrams of a top view ofalternate configurations of the two display HMD of FIGS. 7A and 7Baccording to at least one example embodiment. Like elements arerepresented by like reference numerals, which are described above andmay not be described further with regard to this figure. As shown inFIG. 8, the second display 325 is a flat surface or flat panel displayand the faceplate 510 is formed as an arc or curved surface. Using acurved faceplate 510 in contact with the second display 325 (e.g., thehigh resolution display) can reduce or minimize the image distortionsand image aberrations. Therefore, using the faceplate in addition to thelens 310 (or as an alternative to using the lens 310) can improve theimage displayed by the second display 325.

Further, as shown in FIG. 7A, in an example implementation, the imagecombiner 315 can extend beyond the position of the first display 320 asanother mechanism to minimize visibility of its edges. In other words,extended (in length) image combiner 315 can provide a transition betweenan image projected by the first display 320 and an image projected bythe second display 325. The faceplate 510 can also be a taperedfaceplate. The tapered faceplate may be configured to expand an image toa larger area. Accordingly, the tapered faceplate may enable use of asmaller (e.g., dimensionally or as smaller surface area) second display325 (e.g., a smaller, and potentially less costly, display). The use oftapered faceplate can be used to effect a change in the size of anemissive display.

As shown in FIG. 9, the first display 320 (e.g., the low or mediumresolution display) is shown as an arc over the entire length of thedisplay. In addition, the image combiner 315 mimics the shape of thefirst display 320. In other words, the image combiner 315 is under thefirst display 320 (and, therefore, its full length is not shown). Usinga curved display as the first display 320 in addition to the lens 310(or as an alternative to using the lens 310) can improve the imagedisplayed at the temporal ends of the first display 320. A side view ofFIG. 9 viewed at dotted line 905 (e.g., a slice) could be illustrated asthe combined display system 705 as shown in FIG. 7A.

As shown in FIG. 10, the first display 320 (e.g., the low or mediumresolution display) is shown as a flat panel over a length correspondingto the second display 325 and an arc over a length of the display notcorresponding to the second display 325. In addition, the image combiner315 mimics the shape of the first display 320. In other words, the imagecombiner 315 is under the first display 320 (and, therefore, its fulllength is not shown). Using a curved display approaching the temporalends of the first display 320 in addition to the lens 310 (or as analternative to using the lens 310) can improve the image displayed atthe temporal ends of the first display 320. Further, by including a flatsection of the image combiner 315 can minimize some of the distortionsof an image projected by the second display 325. A side view of FIG. 10viewed at dotted line 1005 (e.g., a slice) could be illustrated as thecombined display system 705 as shown in FIG. 7A. The image combiner 315is shown in FIG. 10 in order to illustrate that the image combiner 315is under the first display 320 (when the HMD is viewed from the top).The ends of the image combiner 315 are shown in the shape of a triangle,however, example embodiments are not limited thereto.

FIG. 11 illustrates a method associated with a two-display HMD accordingto at least one example embodiment. At least one of the steps describedwith regard to FIG. 11 may be performed due to the execution of softwarecode stored in a memory associated with an apparatus (e.g., as shown inFIG. 12) and executed by at least one processor associated with theapparatus. However, alternative embodiments are contemplated such as asystem embodied as a special purpose processor. Although at least one ofthe steps described below are described as being executed by aprocessor, the steps are not necessarily executed by the same processor.In other words, at least one processor may execute at least one of thesteps described below with regard to FIG. 11. Further, the stepsdescribed with regard to FIG. 11 refer to one view or eye (e.g., one ofthe displays viewed by human eyes). However, it is understood that asimilar process is applied to the other view (e.g., the other of thedisplays viewed by human eyes).

As shown in FIG. 11, in step S1105 in a head mounted display, provide afirst display, a second display and an image combiner. For example, asdiscussed above with regard to FIGS. 1, 3A, 3B, 4, 5A, 5B, 6, 7A, 7B and8-10, a HMD can include a first display 320, a second display 325 and animage combiner 315.

In step S1110 project an image of the first display onto the imagecombiner. For example, in a first example implementation, the seconddisplay 325 (e.g., high resolution display) can be projected onto theimage combiner 315. In a second example implementation, the firstdisplay 320 (e.g., low or medium resolution display) can be projectedonto the image combiner 315.

In step S1115 project an image of the second display through the imagecombiner. For example, in the first example implementation, the firstdisplay 320 (e.g., low or medium resolution display) can be projectedthough the image combiner 315. In the second example implementation, thesecond display 325 (e.g., high resolution display) can be projected ontothe image combiner 315.

In step S1120 use the image combiner to direct rays associated with theimage of the first display and associated with the image of the seconddisplay to a lens. For example, the image combiner 315 can reflect theimage projected on to it (e.g., toward the lens 320) causing acombination of this image with the image that is projected through theimage combiner 315. In step S1125 use the lens to adjust a vergence tomake the display of the image viewable in the head mounted display.

Example implementations of a VR system may include an image or videosystem configured to generate images or video (e.g., based on an imageor video source) and a corresponding, at least one, display driverconfigured to control the display of the generated images or video onthe first display 320 and the second display 325. The image or videosystem may be included in the HMD 300 and/or may be associated with anexternal computing device. In some implementations, the display drivermay be included in the HMD 300 and the images or video may be generatedin the external computing device. Further, in some implementations, thedisplay driver may be included in the external computing device and theimages or video may be generated in the external computing device.

FIG. 13 illustrates a technique that uses a light guide 1315 to relayimages through a thin glass or plastic plate (e.g., a thin glass orplastic plate) to create an augmented reality display system. The imageis generated by a high resolution display 325 and a collimator 1305 isused to inject the corresponding light rays into the light guide 1315through a prism 1310. The rays can propagate by internal reflection(e.g., total internal reflection) through the light guide 1315 untilreaching a semitransparent reflector which allows some of the rays tointersect the interior surface of the light guide (e.g., at angles lessthan the critical angle for total internal reflection). Such raysindicated by 1320 then exit the light guide 1315 and are seen by the eye305. There are also techniques that can use holograms or diffractiongratings, combined with internal reflection (e.g., total internalreflection), to create a similar light guide optical system. The distaland proximate surfaces of the light guide 1315 can be transparent.Accordingly, ray 1325 represents rays from a second display (e.g.,display 320), or alternatively, the ambient scene that propagate throughthe transparent light guide 1325 which are also seen by the eye 305.

FIG. 14 illustrates a cross sectional view of another embodiment of anoptical system in which two displays are used per eye to create an imageof high resolution in the central field, and reduced resolution in thefields peripheral the central field. A light guide 1415 is furnishedwith a metallic optical coating on the distal surface obviating the needfor total internal reflection, but also making the system less suited toaugmented reality because propagation of ray 1435 in FIG. 14 can beimpeded. Light guide 1415 is bonded to Fresnel lens 1420 and may bebonded into a recess in Fresnel lens 1420 so that the Fresnel lens 1420and light guide 1415 are integral, so that a smooth interface is formedbetween the distal surface of the light guide and the Fresnel lens. Alower resolution image from display 320 passes through Fresnel lens 1420so that the user is able to view the combined images from display 320and display 325.

As shown in FIG. 14, a high resolution image is created by display 325and light rays from display 325 propagate into collimator 1405.Collimated rays propagate from the collimator 1405 to an injection prism1410, are then relayed through the light guide 1415 and exit toward theeye 305 as represented by ray 1430 which is the chief ray from thecenter of display 325. The image thus formed is a high resolution imagein the field of view of the user.

Another implementation can be made by using total internal reflection tomake a portion of the light guide transparent. This is shown in FIG. 15.In this example, the light guide 1510 is coated with a metallic film1525 and bonded by glue 1530 within a cavity in Fresnel lens 1505. Theglue 1530 and metal film 1525 can be thin (about 10 microns) so that thelight guide 1510 and Fresnel lens 1505 are disposed in close proximity.The metal film 1525 defines a light output area L from which rays 1540from the high resolution display 325 emerge. The thickness of the glue1530 and metal film 1525 can form a gap 1520 meaning that light maypropagate by total internal reflection through the upper light guide,and that the light guide 1510 remains transparent. Light from mediumresolution panoramic display 320 propagates to the Fresnel lens 1505(represented as ray 1545) and the eye 305 sees a combined image from theFresnel lens 1505 and the light guide 1510. FIG. 16 shows a plan view ofthe Fresnel lens 1505 and the light guide 1510. Light from the highresolution display 325 enters the light guide through entrance 1601 andis seen in region 1515 of the light guide 1510 and light from the mediumresolution display 320 is seen elsewhere.

FIG. 12 illustrates a block diagram of a virtual reality VR system 1200associated with a HMD according to at least one example embodiment. Asshown in FIG. 12, the VR system 1200 includes at least one memory 1205and at least one processor 1210. The at least one memory 1205 may beincluded in the HMD (e.g., HMD 300), an external (e.g., external to theHMD) computing device (e.g., a personal computer or a hand held device)and/or included in both the HMD and the external computing device.

The at least one memory includes at least one image/video sourcerepository 1215, a first display driver left eye 1220, a second displaydriver left eye 1225, a first display driver right eye 1230, a seconddisplay driver right eye 1235. These elements may be included in the atleast one memory 1205 of the HMD (e.g., HMD 300), in the at least onememory 1205 of an external (e.g., external to the HMD) computing device(e.g., a personal computer or a hand held device) and/or included in theat least one memory 1205 of both the HMD and the external computingdevice. For example, the at least one image/video source repository 1215may be included in the external device, whereas the other elements areincluded in the HMD. Further, a left eye driver may include the firstdisplay driver left eye 1220 and the second display driver left eye1225; and a right eye driver may include the first display driver righteye 1230 and the second display driver right eye 1235.

The at least one processor 1210 (e.g., a processor formed on a siliconsubstrate, a GaAs substrate, and the like) may be utilized to executeinstructions stored on the at least one memory 1205 (e.g., anon-transitory computer readable medium), so as to thereby implement thevarious features and functions described herein, or additional oralternative features and functions. The at least one processor 1210 andthe at least one memory 1205 may be utilized for various other purposes.For example, the at least one memory 1205 may be understood to representan example of various types of memory and related hardware and softwarewhich might be used to implement any one of the modules describedherein. Systems and/or methods described herein may include data and/orstorage elements. The data and/or storage elements (e.g., data basetables) may be stored in, for example, the at least one memory 1205.

The at least one image/video source repository 1215 may store images andvideos for display on the HMD. (e.g., on the first display 320 and thesecond display 325). The at least one image/video source repository 1215may store images and video corresponding to right eye and left eye(e.g., at different visual perspectives) images and video that can beutilized to generate a three dimensional (3D) image or video. The atleast one image/video source repository 1215 may store raw (e.g.,unformatted) or encoded images and video. Accordingly, the VR system1200 may include (or have access to) a mechanism (e.g., algorithms) toformat and/or decode the images and videos.

The display drivers, the first display driver left eye 1220, the seconddisplay driver left eye 1225, the first display driver right eye 1230,the second display driver right eye 1235, may be configured to controlthe display of an image or video on a corresponding display (e.g., thefirst display 320 or the second display 325). The display drivers may beconfigured to use an image or video as input data and communicatesignals representing color based on the input image or video to thecorresponding display. The signals representing color may be, forexample, correspond to a RGB or YUV format.

The presence of a pixel density may not dictate that the high resolutiondisplay should be operated at high resolution. By combining data onneighboring pixels, the resolution can be blended so that the resolutiondoes not change abruptly at the boundaries of the image of highresolution display. It is also possible to adjust luminance and contrastso that edges are not obvious, so that as the high resolution displayfades out, the low resolution display fades in.

Therefore, the display drivers may operate together to control the firstdisplay 320 and the second display 325 such that the first display 320and the second display 325 are controlled to provide a transition (inaddition to and/or as an alternative to the configuration of the imagecombiner 315) between an image projected by the first display 320 and animage projected by the second display 325. For example, the firstdisplay 320 can be configured to project a progressively darker image(which should not pass through the image combiner 315) from pixels ofthe first display 320 along the boundary of the image combiner 315between the first display 320 and the second display 325 until there isa dark (or no light or black) image at the center of the first display320 corresponding to the center of the image combiner 315. Further, thesecond display 325 can be configured to project a progressively darkerimage (which should not reflect on the image combiner 315) from pixelsof the second display 325 at some threshold distance in from theboundary of the image combiner 315 between the first display 320 and thesecond display 325 toward the center of the second display 325 untilthere is a dark (or no light or black) image at an outer boundary of thesecond display 325.

FIG. 17 illustrates a top view of a block diagram of a portion of a HMDincluding a hybrid lens system according to at least one exampleembodiment. An optical system intended for use over a high field of viewcan also be formed from a hybrid lens system 1705. One way to improveperipheral vision is to employ a curve near the temple as shown in FIG.17. By curving Fresnel lens 1710-1, 1710-2, rays emitted for example bya curved section 1730 of display panel 320 may be collected by hybridoptical system 1705 and relayed to the eye 305. In FIG. 17, the eye 305is turned to a high angle, but peripheral rays may be seen by the retinain peripheral vision without eye rotation, provided the display and lensproduce rays that produce a virtual image in the periphery. Although acurve on one axis is shown, Fresnel lens 1710-1, 1710-2 may be curvedalong multiple axes. In an example implementation, Fresnel lens 1710-1,1710-2 can be formed upon a curved substrate that bends in one or moreaxes. In the illustrated implementation, Fresnel lens 1710-1, 1710-2 isplanar (shown as portion 1710-1) on the nasal side of a central visionlens 1715 but curved along a portion 1710-2 on the temple side of thecentral vision lens 1715.

The hybrid lens system 1705 can include a singlet lens 1715 (e.g.,spherical or aspherical lens) formed at the optical center of theFresnel lens 1710-1, 1710-2 (e.g., as an aspherical Fresnel lens). Inother words, the optical center of singlet lens 1715 can be co-incidentwith the optical center of Fresnel lens 1710-1, 1710-2. In theillustrated implementation, the optical centers of the lenses are placedin front of the user's pupil/cornea (when the eye is gazing straightahead). In the illustrated implementation, the singlet lens 1715 (e.g.,as a central vision lens) can be implemented as a refractive lens havingtwo curved surfaces. However, a refractive lens having only a singlecurved surface may also be utilized. Furthermore, the singlet lens 1715may be implemented as a progressive lens, an achromatic lens, adiffractive optical element, a hologram, or even a hybridrefractive-diffractive lens.

The display 320 can include a flat section 1725 and curved section 1730that extends beyond the curved section 1720 of hybrid lens system 1705so that display 320 may be viewed directly without any lens (asillustrated by arrow 1735), or through clear plastic. In such anembodiment, display 320 can be extended to a peripheral region of theFOV where the eye has little peripheral resolution such that merelymotion and awareness of light can be perceived. In this peripheralregion (outside of lens 1705) no lensing may be necessary. This isanalogous to the image presented by prescription eyewear when lookingbeyond the outer edge of the lens. In such a case the peripheral imageis out of focus, which may not detract from a feeling of presence.

FIGS. 18 and 19 illustrate views of a line diagram of a HMD according toat least one example embodiment. As shown in FIG. 18, the HMD includes afirst display 1805-1, 1805-2 (or display portion) and a second display1810-1, 1810-2 (or display portion). The first display 1805-1, 1805-2can have a higher pixel density than the second display 1810-1, 1810-2.Although the implementation illustrated in FIGS. 18 and 19 is shown astwo displays, the first display 1805-1, 1805-2 can be a single displaywith two portions or the first display 1805-1, 1805-2 can be twoseparate displays. The second display 1810-1, 1810-2 can be a singledisplay, a single display with two or more portions and/or two or moredisplays. The second display 1810-1, 1810-2 can include at least onecurved portion.

The HMD can include at least one image combiner 1815-1, 1815-2configured to combine two images. For example the at least one imagecombiner 1815-1, 1815-2 can reflect an image projected by the firstdisplay 1805-1, 1805-2 and allow an image projected by the seconddisplay 1810-1, 1810-2 to pass through the at least one image combiner1815-1, 1815-2.

The HMD can include at least one lens 1820-1, 1820-2. The at least onelens 1820-1, 1820-2 can be a hybrid lens system (e.g., hybrid lenssystem 1705). Therefore, the at least one lens 1820-1, 1820-2 caninclude a singlet lens 1905 shown in FIG. 19. The HMD can include anenclosure 1835 configured to enclose the elements (e.g., first display1805-1, 1805-2) of the HMD. The HMD can include a frame (not shown) towhich elements of the HMD may be disposed on and/or fastened to. Forexample, the frame and a bracket 1825 together may fasten the firstdisplay 1805-1 in a position within the HMD.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.Various implementations of the systems and techniques described here canbe realized as and/or generally be referred to herein as a circuit, amodule, a block, or a system that can combine software and hardwareaspects. For example, a module may include the functions/acts/computerprogram instructions executing on a processor (e.g., a processor formedon a silicon substrate, a GaAs substrate, and the like) or some otherprogrammable data processing apparatus.

Some of the above example embodiments are described as processes ormethods depicted as flowcharts. Although the flowcharts describe theoperations as sequential processes, many of the operations may beperformed in parallel, concurrently or simultaneously. In addition, theorder of operations may be re-arranged. The processes may be terminatedwhen their operations are completed, but may also have additional stepsnot included in the figure. The processes may correspond to methods,functions, procedures, subroutines, subprograms, etc.

Methods discussed above, some of which are illustrated by the flowcharts, may be implemented by hardware, software, firmware, middleware,microcode, hardware description languages, or any combination thereof.When implemented in software, firmware, middleware or microcode, theprogram code or code segments to perform the necessary tasks may bestored in a machine or computer readable medium such as a storagemedium. A processor(s) may perform the necessary tasks.

Specific structural and functional details disclosed herein are merelyrepresentative for purposes of describing example embodiments. Exampleembodiments, however, be embodied in many alternate forms and should notbe construed as limited to only the embodiments set forth herein.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first element could be termed asecond element, and, similarly, a second element could be termed a firstelement, without departing from the scope of example embodiments. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, when an element is referred to as being “directlyconnected” or “directly coupled” to another element, there are nointervening elements present. Other words used to describe therelationship between elements should be interpreted in a like fashion(e.g., “between” versus “directly between,” “adjacent” versus “directlyadjacent,” etc.).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of exampleembodiments. As used herein, the singular forms “a,” “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises,” “comprising,” “includes” and/or “including,” when usedherein, specify the presence of stated features, integers, steps,operations, elements and/or components, but do not preclude the presenceor addition of one or more other features, integers, steps, operations,elements, components and/or groups thereof.

It should also be noted that in some alternative implementations, thefunctions/acts noted may occur out of the order noted in the figures.For example, two figures shown in succession may in fact be executedconcurrently or may sometimes be executed in the reverse order,depending upon the functionality/acts involved.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which example embodiments belong. Itwill be further understood that terms, e.g., those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art andwill not be interpreted in an idealized or overly formal sense unlessexpressly so defined herein.

Portions of the above example embodiments and corresponding detaileddescription are presented in terms of software, or algorithms andsymbolic representations of operation on data bits within a computermemory. These descriptions and representations are the ones by whichthose of ordinary skill in the art effectively convey the substance oftheir work to others of ordinary skill in the art. An algorithm, as theterm is used here, and as it is used generally, is conceived to be aself-consistent sequence of steps leading to a desired result. The stepsare those requiring physical manipulations of physical quantities.Usually, though not necessarily, these quantities take the form ofoptical, electrical, or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

In the above illustrative embodiments, reference to acts and symbolicrepresentations of operations (e.g., in the form of flowcharts) that maybe implemented as program modules or functional processes includeroutines, programs, objects, components, data structures, etc., thatperform particular tasks or implement particular abstract data types andmay be described and/or implemented using existing hardware at existingstructural elements. Such existing hardware may include one or moreCentral Processing Units (CPUs), digital signal processors (DSPs),application-specific-integrated-circuits, field programmable gate arrays(FPGAs) computers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise, or as is apparent from the discussion,terms such as “processing” or “computing” or “calculating” or“determining” of “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical, electronicquantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

Note also that the software implemented aspects of the exampleembodiments are typically encoded on some form of non-transitory programstorage medium or implemented over some type of transmission medium. Theprogram storage medium may be magnetic (e.g., a floppy disk or a harddrive) or optical (e.g., a compact disk read only memory, or “CD ROM”),and may be read only or random access. Similarly, the transmissionmedium may be twisted wire pairs, coaxial cable, optical fiber, or someother suitable transmission medium known to the art. The exampleembodiments not limited by these aspects of any given implementation.

Lastly, it should also be noted that whilst the accompanying claims setout particular combinations of features described herein, the scope ofthe present disclosure is not limited to the particular combinationshereafter claimed, but instead extends to encompass any combination offeatures or embodiments herein disclosed irrespective of whether or notthat particular combination has been specifically enumerated in theaccompanying claims at this time.

What is claimed is:
 1. A head mounted display (HMD) comprising: a firstdisplay portion included in the HMD, the first display portion having afirst pixel density; a second display portion included in the HMD, thesecond display portion having the first pixel density; a third displayportion attached to the HMD, the third display portion having a secondpixel density; and at least one image combiner configured to opticallycombine two images by reflecting light rays representing an imageprojected by the first display portion and the second display portionand allowing light rays representing an image projected by the thirddisplay portion to pass through the at least one image combiner, the atleast one image combiner including a region configured as a transitionbetween the image projected by the first display portion and the seconddisplay portion and the image projected by the third display portion. 2.The HMD of claim 1, wherein the at least one image combiner is furtherconfigured to block light rays representing a portion of the imageprojected by the third display portion, the light rays representingportion of the image projected by the third display portioncorresponding to the light rays representing the image projected by thefirst display portion and the second display portion.
 3. The HMD ofclaim 1, wherein the at least one image combiner is further configuredto block light rays representing a portion of the image projected by thefirst display portion and the second display portion, the light raysrepresenting the portion of the image projected by the first displayportion and the second display portion corresponding to the light raysrepresenting the image projected by the third display portion.
 4. TheHMD of claim 1, wherein the first pixel density is a higher pixeldensity than the second pixel density, and the third display portion isconfigured to reduce a brightness of a portion of the image projected bythe third display portion, the portion of the image projected withreduced brightness by the third display portion corresponding to theimage projected by the first display portion and the second displayportion.
 5. The HMD of claim 1, wherein the first pixel density is alower pixel density than the second pixel density, and the first displayportion and the second display portion are configured to reduce abrightness of a portion of the image projected by the first displayportion and the second display portion, the portion of the imageprojected with reduced brightness by the first display portion and thesecond display portion corresponding to the image projected by the thirddisplay portion.
 6. The HMD of claim 1, wherein the two images areblended at a boundary between the two images.
 7. The HMD of claim 1,wherein the HMD is communicatively coupled to a computing deviceconfigured to generate the two images using an optical fiber.
 8. The HMDof claim 1, wherein the first pixel density is a higher pixel densitythan the second pixel density, and the first display portion and thesecond display portion are positioned above the third display portion.9. The HMD of claim 1, wherein at least one of the first displayportion, the second display portion and the third display portioninclude a curved portion.
 10. The HMD of claim 1, further comprising atleast one lens.
 11. The HMD of claim 1, further comprising at least oneof a motion sensor and an eye tracking component each configured todetect a change in a view position of a user of the HMD.
 12. A headmounted display (HMD) of a virtual reality (VR) system, the HMDcomprising: a first combined display system configured to receive afirst image, the first combined display system including: a firstdisplay portion configured to project a first portion of the firstimage, a second display portion configured to project a second portionof the first image, and an image combiner configured to opticallycombine two images by reflecting light rays representing the firstportion of the first image and allowing light rays representing thesecond portion of the first image to pass through the image combiner,the image combiner including a region configured as a transition betweenthe first portion of the first image and the second portion of the firstimage; and a second combined display system configured to receive asecond image, the second image being a different view perspective of thefirst image, the second combined display system including: a thirddisplay portion configured to project a first portion of the secondimage, a fourth display portion configured to project a second portionof the second image, and an image combiner configured to opticallycombine two images by reflecting light rays representing the firstportion of the second image and allowing light rays representing thesecond portion of the second image to pass through the image combiner,the image combiner including a region configured as a transition betweenthe first portion of the second image and the second portion of thesecond image.
 13. The HMD of claim 12, wherein the HMD iscommunicatively coupled to a computing device associated with an imagerepository via an optical fiber.
 14. The HMD of claim 12, wherein thefirst display portion and the second display portion have a first pixeldensity, the third display portion and the fourth display portion have asecond pixel density, the second pixel density being lower than thefirst pixel density, and the third display portion and the fourthdisplay portion include a curved portion.
 15. The HMD of claim 12,wherein the first display portion and the second display portion have afirst pixel density, the third display portion and the fourth displayportion have a second pixel density, the second pixel density beinglower than the first pixel density, and the first display portion andthe second display portion are positioned above the third displayportion and the fourth display portion.
 16. The HMD of claim 12, furthercomprising at least one lens.
 17. The HMD of claim 12, wherein the twoimages are blended at a boundary between the two images.
 18. The HMD ofclaim 12, wherein the first display portion is configured to reduce abrightness of a portion of the first portion of the first image, and thethird display portion is configured to reduce a brightness of a portionof the first portion of the second image.
 19. The HMD of claim 12,wherein the second display portion is configured to reduce a brightnessof a portion of the second portion of the first image, and the fourthdisplay portion is configured to reduce a brightness of a portion of thesecond portion of the second image.